{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "df7d42b9-58a6-434c-a2d7-0b61142f6d3e",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_position: 0\n",
    "---\n",
    "```{=mdx}\n",
    "import CodeBlock from \"@theme/CodeBlock\";\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2195672-0cab-4967-ba8a-c6544635547d",
   "metadata": {},
   "source": [
    "# Quickstart\n",
    "\n",
    "This example will show how to use query analysis in a basic end-to-end example. This will cover creating a simple index, showing a failure mode that occur when passing a raw user question to that index, and then an example of how query analysis can help address that issue. There are MANY different query analysis techniques and this end-to-end example will not show all of them.\n",
    "\n",
    "For the purpose of this example, we will do retrieval over the LangChain YouTube videos."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4079b57-4369-49c9-b2ad-c809b5408d7e",
   "metadata": {},
   "source": [
    "## Setup\n",
    "#### Install dependencies\n",
    "\n",
    "```{=mdx}\n",
    "import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n",
    "import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
    "\n",
    "<IntegrationInstallTooltip></IntegrationInstallTooltip>\n",
    "\n",
    "<Npm2Yarn>\n",
    "  langchain @langchain/community @langchain/openai youtubei.js chromadb youtube-transcript\n",
    "</Npm2Yarn>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79d66a45-a05c-4d22-b011-b1cdbdfc8f9c",
   "metadata": {},
   "source": [
    "#### Set environment variables\n",
    "\n",
    "We'll use OpenAI in this example:\n",
    "\n",
    "```env\n",
    "OPENAI_API_KEY=your-api-key\n",
    "\n",
    "# Optional, use LangSmith for best-in-class observability\n",
    "LANGSMITH_API_KEY=your-api-key\n",
    "LANGCHAIN_TRACING_V2=true\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c20b48b8-16d7-4089-bc17-f2d240b3935a",
   "metadata": {},
   "source": [
    "### Load documents\n",
    "\n",
    "We can use the `YouTubeLoader` to load transcripts of a few LangChain videos:\n",
    "\n",
    "```{=mdx}\n",
    "import LoadYtVideos from \"@examples/use_cases/query_analysis/quickstart/load_yt_videos.ts\";\n",
    "\n",
    "<CodeBlock language=\"typescript\">{LoadYtVideos}</CodeBlock>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "516b47fb",
   "metadata": {},
   "source": [
    "Here's the metadata associated with each video.\n",
    "\n",
    "We can see that each document also has a title, view count, publication date, and length:\n",
    "\n",
    "```{=mdx}\n",
    "import VideoMetadata from \"@examples/use_cases/query_analysis/quickstart/metadata.ts\";\n",
    "\n",
    "<CodeBlock language=\"typescript\">{VideoMetadata}</CodeBlock>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "561697c8-b848-4b12-847c-ab6a8e2d1ae6",
   "metadata": {},
   "source": [
    "### Indexing documents\n",
    "\n",
    "Whenever we perform retrieval we need to create an index of documents that we can query. We'll use a vector\n",
    "store to index our documents, and we'll chunk them first to make our retrievals more concise and precise:\n",
    "\n",
    "```{=mdx}\n",
    "import IndexDocs from \"@examples/use_cases/query_analysis/quickstart/index_docs.ts\";\n",
    "\n",
    "<CodeBlock language=\"typescript\">{IndexDocs}</CodeBlock>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "850717b9",
   "metadata": {},
   "source": [
    "Then later, you can retrieve the index without having to re-query and embed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "247fae46",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Module: null prototype] {\n",
       "  AdminClient: \u001b[36m[class AdminClient]\u001b[39m,\n",
       "  ChromaClient: \u001b[36m[class ChromaClient]\u001b[39m,\n",
       "  CloudClient: \u001b[36m[class CloudClient extends ChromaClient]\u001b[39m,\n",
       "  CohereEmbeddingFunction: \u001b[36m[class CohereEmbeddingFunction]\u001b[39m,\n",
       "  Collection: \u001b[36m[class Collection]\u001b[39m,\n",
       "  DefaultEmbeddingFunction: \u001b[36m[class _DefaultEmbeddingFunction]\u001b[39m,\n",
       "  GoogleGenerativeAiEmbeddingFunction: \u001b[36m[class _GoogleGenerativeAiEmbeddingFunction]\u001b[39m,\n",
       "  HuggingFaceEmbeddingServerFunction: \u001b[36m[class HuggingFaceEmbeddingServerFunction]\u001b[39m,\n",
       "  IncludeEnum: {\n",
       "    Documents: \u001b[32m\"documents\"\u001b[39m,\n",
       "    Embeddings: \u001b[32m\"embeddings\"\u001b[39m,\n",
       "    Metadatas: \u001b[32m\"metadatas\"\u001b[39m,\n",
       "    Distances: \u001b[32m\"distances\"\u001b[39m\n",
       "  },\n",
       "  JinaEmbeddingFunction: \u001b[36m[class JinaEmbeddingFunction]\u001b[39m,\n",
       "  OpenAIEmbeddingFunction: \u001b[36m[class _OpenAIEmbeddingFunction]\u001b[39m,\n",
       "  TransformersEmbeddingFunction: \u001b[36m[class _TransformersEmbeddingFunction]\u001b[39m\n",
       "}"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import \"chromadb\";\n",
    "import { OpenAIEmbeddings } from \"@langchain/openai\";\n",
    "import { Chroma } from \"@langchain/community/vectorstores/chroma\";\n",
    "\n",
    "const embeddings = new OpenAIEmbeddings({\n",
    "  modelName: \"text-embedding-3-small\"\n",
    "});\n",
    "const vectorStore = await Chroma.fromExistingCollection(embeddings, {\n",
    "  collectionName: \"yt-videos\",\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "483d8d0a-5c1b-46b0-862c-a4eccfd5ae3c",
   "metadata": {},
   "source": [
    "## Retrieval without query analysis\n",
    "\n",
    "We can perform similarity search on a user question directly to find chunks relevant to the question:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "09435e9b-57b4-41b1-b34a-449815bdfae0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenGPTs\n",
      "hardcoded that it will always do a retrieval step here the assistant decides whether to do a retrieval step or not sometimes this is good sometimes this is bad sometimes it you don't need to do a retrieval step when I said hi it didn't need to call it tool um but other times you know the the llm might mess up and not realize that it needs to do a retrieval step and so the rag bot will always do a retrieval step so it's more focused there because this is also a simpler architecture so it's always\n"
     ]
    }
   ],
   "source": [
    "const searchResults = await vectorStore.similaritySearch(\"how do I build a RAG agent\");\n",
    "console.log(searchResults[0].metadata.title);\n",
    "console.log(searchResults[0].pageContent.slice(0, 500));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a79ef1b-7edd-4b68-98e5-c0e4c0dd02e6",
   "metadata": {},
   "source": [
    "This works pretty okay! Our first result is somewhat relevant to the question.\n",
    "\n",
    "What if we wanted to search for results from a specific time period?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7adbfc11-ca01-4883-8978-e4f6e4a1d23d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenGPTs\n",
      "2024\n",
      "hardcoded that it will always do a retrieval step here the assistant decides whether to do a retrieval step or not sometimes this is good sometimes this is bad sometimes it you don't need to do a retrieval step when I said hi it didn't need to call it tool um but other times you know the the llm might mess up and not realize that it needs to do a retrieval step and so the rag bot will always do a retrieval step so it's more focused there because this is also a simpler architecture so it's always\n"
     ]
    }
   ],
   "source": [
    "const searchResults = await vectorStore.similaritySearch(\"videos on RAG published in 2023\");\n",
    "console.log(searchResults[0].metadata.title);\n",
    "console.log(searchResults[0].metadata.publish_year);\n",
    "console.log(searchResults[0].pageContent.slice(0, 500));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4790e2db-3c6e-440b-b6e8-ebdd6600fda5",
   "metadata": {},
   "source": [
    "Our first result is from 2024, and not very relevant to the input. Since we're just searching against document contents, there's no way for the results to be filtered on any document attributes.\n",
    "\n",
    "This is just one failure mode that can arise. Let's now take a look at how a basic form of query analysis can fix it!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57396e23-c192-4d97-846b-5eacea4d6b8d",
   "metadata": {},
   "source": [
    "## Query analysis\n",
    "\n",
    "To handle these failure modes we'll do some query structuring. This will involve defining a **query schema** that contains some date filters and use a function-calling model to convert a user question into a structured queries. \n",
    "\n",
    "### Query schema\n",
    "In this case we'll have explicit min and max attributes for publication date so that it can be filtered on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "0b51dd76-820d-41a4-98c8-893f6fe0d1ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "import { z } from 'zod';\n",
    "\n",
    "const searchSchema = z.object({\n",
    "  query: z.string().describe(\"Similarity search query applied to video transcripts.\"),\n",
    "  publish_year: z.number().optional().describe(\"Year of video publication.\"),\n",
    "}).describe(\"Search over a database of tutorial videos about a software library.\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8b08c52-1ce9-4d8b-a779-cbe8efde51d1",
   "metadata": {},
   "source": [
    "### Query generation\n",
    "\n",
    "To convert user questions to structured queries we'll make use of OpenAI's function-calling API. Specifically we'll use the new [ChatModel.withStructuredOutput()](https://api.js.langchain.com/classes/langchain_core_language_models_base.BaseLanguageModel.html#withStructuredOutput) constructor to handle passing the schema to the model and parsing the output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "783c03c3-8c72-4f88-9cf4-5829ce6745d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import { ChatPromptTemplate } from \"@langchain/core/prompts\";\n",
    "import { ChatOpenAI } from \"@langchain/openai\";\n",
    "import { RunnablePassthrough, RunnableSequence } from \"@langchain/core/runnables\";\n",
    "\n",
    "const system = `You are an expert at converting user questions into database queries.\n",
    "You have access to a database of tutorial videos about a software library for building LLM-powered applications.\n",
    "Given a question, return a list of database queries optimized to retrieve the most relevant results.\n",
    "\n",
    "If there are acronyms or words you are not familiar with, do not try to rephrase them.`;\n",
    "const prompt = ChatPromptTemplate.fromMessages([\n",
    "  [\"system\", system],\n",
    "  [\"human\", \"{question}\"]\n",
    "]);\n",
    "const llm = new ChatOpenAI({\n",
    "  modelName: \"gpt-3.5-turbo-0125\",\n",
    "  temperature: 0,\n",
    "});\n",
    "const structuredLLM = llm.withStructuredOutput(searchSchema, {\n",
    "  name: \"search\",\n",
    "});\n",
    "\n",
    "const queryAnalyzer = RunnableSequence.from([\n",
    "  {\n",
    "    question: new RunnablePassthrough(),\n",
    "  },\n",
    "  prompt,\n",
    "  structuredLLM,\n",
    "]);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f403517a-b8e3-44ac-b0a6-02f8305635a2",
   "metadata": {},
   "source": [
    "Let's see what queries our analyzer generates for the questions we searched earlier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "bc1d3863",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{ query: \"build a rag agent\" }\n"
     ]
    }
   ],
   "source": [
    "console.log(await queryAnalyzer.invoke(\"How do I build a rag agent\"));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "af62af17-4f90-4dbd-a8b4-dfff51f1db95",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{ query: \"RAG\", publish_year: 2023 }\n"
     ]
    }
   ],
   "source": [
    "console.log(await queryAnalyzer.invoke(\"videos on RAG published in 2023\"));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7c65b2f-7881-45fc-a47b-a4eaaf48245f",
   "metadata": {},
   "source": [
    "## Retrieval with query analysis\n",
    "\n",
    "Our query analysis looks pretty good; now let's try using our generated queries to actually perform retrieval. \n",
    "\n",
    "**Note:** in our example, we specified `tool_choice: \"Search\"`. This will force the LLM to call one - and only one - function, meaning that we will always have one optimized query to look up. Note that this is not always the case - see other guides for how to deal with situations when no - or multiple - optimized queries are returned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "8dac7866",
   "metadata": {},
   "outputs": [],
   "source": [
    "import { DocumentInterface } from \"@langchain/core/documents\";\n",
    "\n",
    "const retrieval = async (input: { query: string, publish_year?: number }): Promise<DocumentInterface[]> => {\n",
    "    let _filter: Record<string, any> = {};\n",
    "    if (input.publish_year) {\n",
    "      // This syntax is specific to Chroma\n",
    "      // the vector database we are using.\n",
    "      _filter = {\n",
    "        publish_year: {\n",
    "          \"$eq\": input.publish_year\n",
    "        }\n",
    "      };\n",
    "    }\n",
    "  \n",
    "    return vectorStore.similaritySearch(input.query, undefined, _filter);\n",
    "  };"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "232ad8a7-7990-4066-9228-d35a555f7293",
   "metadata": {},
   "outputs": [],
   "source": [
    "import { RunnableLambda } from \"@langchain/core/runnables\";\n",
    "\n",
    "const retrievalChain = queryAnalyzer.pipe(new RunnableLambda({\n",
    "  func: async (input) => retrieval(input as unknown as { query: string, publish_year?: number })\n",
    "}));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6a4460c",
   "metadata": {},
   "source": [
    "We can now run this chain on the problematic input from before, and see that it yields only results from that year!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e7f683b5-b1c5-4dec-b163-2efc162a2b51",
   "metadata": {},
   "outputs": [],
   "source": [
    "const results = await retrievalChain.invoke(\"RAG tutorial published in 2023\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "1ad52512-b3e8-42a3-8701-d9e87fb8b46c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    title: \"Getting Started with Multi-Modal LLMs\",\n",
      "    year: \"2023-12-20T08:00:00.000Z\"\n",
      "  },\n",
      "  {\n",
      "    title: \"LangServe and LangChain Templates Webinar\",\n",
      "    year: \"2023-11-02T07:00:00.000Z\"\n",
      "  },\n",
      "  {\n",
      "    title: \"Getting Started with Multi-Modal LLMs\",\n",
      "    year: \"2023-12-20T08:00:00.000Z\"\n",
      "  },\n",
      "  {\n",
      "    title: \"Building a Research Assistant from Scratch\",\n",
      "    year: \"2023-11-16T08:00:00.000Z\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "console.log(results.map((doc) => ({ title: doc.metadata.title, year: doc.metadata.publish_date })));"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Deno",
   "language": "typescript",
   "name": "deno"
  },
  "language_info": {
   "file_extension": ".ts",
   "mimetype": "text/x.typescript",
   "name": "typescript",
   "nb_converter": "script",
   "pygments_lexer": "typescript",
   "version": "5.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
